Systems such as personal computers (PC's) often use small printed-circuit board (PCB) daughter cards known as memory modules instead of directly mounting individual memory chips on a motherboard. The memory modules are built to meet specifications set by industry standards, thus ensuring a wide potential market. High-volume production and competition have driven module costs down dramatically, benefiting the PC buyer.
Memory modules are made in many different sizes and capacities, such as older 30-pin and 72-pin single-inline memory modules (SIMMs) and newer 168-pin, 184-pin, and 240-pin dual inline memory modules (DIMMs). The “pins” were originally pins extending from the module's edge, but now most modules are leadless, having metal contact pads or leads. The modules are small in size, being about 3-5 inches long and about an inch to an inch and a half in height.
The memory modules contain a small printed-circuit board substrate, typically a multi-layer board with alternating laminated layers of fiberglass insulation and foil or metal interconnect layers. Surface mounted components such as DRAM chips and capacitors are soldered onto one or both surfaces of the substrate.
FIG. 1 shows a fully-buffered memory module. Memory module 10 contains a substrate such as a multi-layer printed-circuit board (PCB) with surface-mounted DRAM chips 22 mounted to the front surface or side of the substrate, as shown in FIG. 1, while more DRAM chips 22 are mounted to the back side or surface of the substrate (not shown). Memory module 10 is a fully-buffered dual-inline memory module (FB-DIMM) that is fully buffered by Advanced Memory Buffer (AMB) 24 on memory module 10.
Metal contact pads 12 are positioned along the bottom edge of the module on both front and back surfaces. Metal contact pads 12 mate with pads on a module socket to electrically connect the module to a PC's motherboard. Holes 16 are present on some kinds of modules to ensure that the module is correctly positioned in the socket. Notches 14 also ensure correct insertion of the module. Capacitors or other discrete components are surface-mounted on the substrate to filter noise from the DRAM chips 22.
As system clock speeds increase, data must be transmitted and received at ever-increasing rates. Differential signaling techniques are being used to carry data, clock, and commands to and from memory modules. AMB 24 is a chip mounted onto the substrate of memory module 10 to support differential signaling through metal contact pads 12. AMB 24 sends and receives external packets or frames of data and commands to other memory modules in other sockets over differential data lines in metal contact pads 12.
AMB 24 also extracts data from the external frames and writes the extracted data to DRAM chips 22 on memory module 10. Command frames to read data are decoded by AMB 24. AMB 24 sends addresses and read signals to DRAM chips 22 to read the requested data, and packages the data into external frames that are transmitted from AMB 24 over metal contact pads 12 to other memory modules in a serial daisy chain and eventually to the host processor.
Memory module 10 is known as a fully-buffered memory module since AMB 24 buffers data from DRAM chips 22 to metal contact pads 12. DRAM chips 22 do not send and receive data directly from metal contact pads 12 as in many prior memory module standards. Since DRAM chips 22 do not directly communicate data with metal contact pads 12, signals on metal contact pads 12 can operate at very high data rates.
FIG. 2 shows detail of an advanced memory buffer on a fully-buffered memory module. AMB 24 contains DRAM controller 50, which generates DRAM control signals to read and write data to and from DRAM chips 22 on memory module 10. Data is temporarily stored in FIFO 58 during transfers.
The data from FIFO 58 is encapsulated in frames that are sent over differential signals through metal contact pads 12. Rather than being sent directly to the host central processing unit (CPU), the frames are passed from one memory module to the next memory module, along a daisy chain series of memory modules, until the frame reaches the host CPU. Differential data lines in the direction toward the host CPU are known as northbound lanes, while differential data lines from the CPU toward the memory modules are known as southbound lanes.
When a frame is sent from the host CPU toward a memory module, the frame is sent over the southbound lanes toward one of the memory modules in the daisy chain. Each memory module passes the frame along to the next memory module in the daisy chain. Southbound lanes that are input to a memory module are buffered by its AMB 24 using re-timing and re-synchronizing buffers 54. Re-timing and re-synchronizing buffers 54 restore the timing of the differential signals prior to retransmission. Input buffers 52 and output buffers 56 contain differential receivers and transmitters for the southbound lanes that are buffered by re-timing and re-synchronizing buffers 54.
Frames that are destined for the current memory module are copied into FIFO 58 and processed by AMB 24. For example, for a write frame, the data from FIFO 58 is written to DRAM chips 22 on the memory module by AMB 24. For a read, the data read from DRAM chips 22 is stored in FIFO 58. AMB 24 forms a frame and sends the frame to northbound re-timing and re-synchronizing buffers 64 and out over the northbound lanes from differential output buffer 62. Input buffers 66 and output buffers 62 contain differential receivers and transmitters for the northbound lanes that are buffered by re-timing and re-synchronizing buffers 64.
Forming outgoing frames and examining packet headers of incoming frames is performed by packet controller 60. Packet controller 60 may support a variety of frame sizes, formats, and features that may be programmable.
FIG. 3 shows fully-buffered memory modules daisy chained together in a series. Host CPU 210 on motherboard 28 reads and writes main memory in DRAM chips 22 on memory modules 201-204 through memory controller 220 on motherboard 28. Memory modules 201-204 are inserted into memory module sockets on motherboard 28.
Rather than read and write DRAM chips 22 directly, host CPU 210 sends read and write commands in packets or frames that are sent over southbound lanes 102. The frame from host CPU 210 is first sent from memory controller 220 to first memory module 201 in the first socket. AMB 24 on first memory module 201 examines the frame to see if it is intended for first memory module 201 and re-buffers and passes the frame on to second memory module 202 over another segment of southbound lanes 102. AMB 24 on second memory module 202 examines the frame and passes the frame on to third memory module 203. AMB 24 on third memory module 203 examines the frame and passes the frame on to fourth memory module 204.
When data is read, or a reply frame is sent back to host CPU 210, northbound lanes 104 are used. For example, when DRAM chips 22 on third memory module 203 are read, the read data is packaged in a frame by AMB 24 and sent over northbound lanes 104 to second memory module 202, which re-buffers the frame and sends it over another segment of northbound lanes 104 to first memory module 201. First memory module 201 then re-buffers the frame of data and sends it over northbound lanes 104 to memory controller 220 and on to host CPU 210.
Since northbound lanes 104 and southbound lanes 102 are composed of many point-to-point links between adjacent memory modules, the length and loading of these segments is reduced, allowing for higher speed signaling. Signaling is to AMB 24 on each memory module rather than to DRAM chips 22.
When branches are included in a physical bus link, there are 3 or more endpoints. Reflections may occur from the third endpoint (stub) and create distortions for signals being sent between the other two endpoints. Terminating the branching physical bus link is more difficult and less effective than when a physical bus link has only two endpoints. Thus southbound lanes 102 and northbound lanes 104 are composed of physical bus links having only 2 endpoints. A serial chain of fully-buffered memory modules has been the most obvious and widely used topology since it is a natural extension of the 2-endpoint physical links.
While such a daisy chain of fully-buffered memory modules is useful, memory modules at the end of a long serial chain of memory modules have increased delays or latencies for data to and from the CPU at the head of the chain. The data has to pass through and be buffered and re-timed by each of the intervening memory modules, resulting in significant delays. It is desirable to reduce such delays by improving the memory-bus topology.